The present application relates generally to an improved data processing apparatus and method and more specifically to mechanisms for performing deep embedding for natural language content based on identified semantic dependencies.
Word embedding is the collective name for a set of language modeling and feature learning techniques in natural language processing (NLP) where words or phrases from a vocabulary are mapped to vectors of real numbers. Word embedding is used by NLP systems as one mechanism for reasoning over natural language sentences. Without word embedding, an NLP system operates on strings of characters, similar groups of words can be considered differently by the NLP system. For example, “The President of the United States visited New York City last week” and “Mr. Trump came to NYC on May 4” have high semantic similarity, but low string similarity. Thus, through word embedding, these sentences may be translated into vectors, e.g., [0.92, 0.1, . . . , 0.1] and [0.91, 0.1, . . . , 0.1], such that their semantic similarity becomes much more readily apparent to a NLP system when comparing the vector representations.